1. Field of the Invention
The present invention relates to an apparatus for reconstructing an image of an arbitrary view point (or an arbitrary point of view (POV)) from plural photographed images.
2. Related Background Art
There have been attempts to generate an image of an arbitrary view point from photographed images. For example there is employed a method of determining a shape of an object from two or more images utilizing a stereo measurement, and generating an image of an arbitrary view point by a computer graphics (CG) technology. In such method, however, it is necessary to identify corresponding points in two or more images, and such operation is very difficult under certain conditions thereby resulting in a low image quality of a generated image. In contrast, an image-based rendering (IBR) technology, particularly a method of obtaining a light beam coming into the space and generating an image of an arbitrary view point utilizing such beam, has an advantage capable in principle of generating a proper image of an arbitrary view point by measuring a position and an orientation of a camera at an image pickup (image being a record of a ray passing through a lens of the camera).
FIG. 31A is a schematic view showing a CCD camera in a state of image pickup of an object. An ordinary image pickup plane is a CCD, but, for the simplicity of explanation, an imaginary plane folded back symmetrically with respect to a center of a lens is assumed as an image pickup plane, and, for further simplifying the explanation, the image pickup plane is assumed to be constituted of a one-dimensional line sensor. As shown in FIG. 31B, in case a multi view point image can be obtained at an arbitrary view point on a line L, a color of a pixel i of an image observable at a view point position Q is same as a color of a pixel i′ of an image having a view point position at a crossing point R of a line connecting the pixel i and the view point position Q and the line L. It is thus possible to reconstruct an image at a completely new view point position at which the image pickup has not been made, by determining such relationship for all the pixels of the image to be reconstructed. In “Light Field Rendering” (M. Levoy and P. Hanrahan, Proc. SIGGRAPH 96, p. 31-42, 1996), which is a representative example of such method, the light beams are acquired by photographing images with cameras positioned in lattice points as shown in FIG. 32.
However, in case of an image pickup apparatus such as a digital camera utilizing the image-based rendering technology, it is not easy to set a view point position of a reconstructed image in an actual space.